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ABSTRACT 



This paper discusses the challenges that the digital age 
poses for recordkeeping and the National Archives of Australia's response to 
these challenges. The Archive's development of inter-dependent standards, 
policies, and guidelines for government recordkeeping is summarized. An 
initiative for developing and promoting resource discovery metadata and 
recordkeeping metadata that resulted in the Australian Government Locator 
Service (AGLS ) metadata standard is described. The set of 19 AGLS resource 
discovery elements are listed according to their descriptive purpose, 
including four AGLS elements (function, availability, mandate, and audience) 
that were added to the basic elements of Dublin Core. An overview is 
presented of the Australian Governments Interactive Functions Thesaurus 
(AGIFT) , recommended for use when creating AGLS metadata. The purposes of 
recordkeeping metadata are summarized, and the relationship between the eight 
mandatory elements of the recordkeeping metadata standard (agent, rights 
management, title, date, aggregation level, record identifier, management 
history, and disposal) and the elements of the AGLS standard are shown. 
Finally, a systematic approach to the effective management of World Wide 
Web-based records is highlighted. (Contains 15 references.) (MES) 
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Enabling Best Practice Recordkeeping in the Digital Age 

Anne Robertson 

Thank you for the opportunity to talk with you this morning about the challenges that the digital 
age poses for recordkeeping and the National Archives of Australia's response to these challenges. 

Most of you would be familiar with the Archives as the custodian of the significant records of the 
Commonwealth Government. The records in our collection trace the events and decisions that have 
shaped our nation over the past one hundred years. We hold the papers of Governors-General, 
Prime Ministers and Ministers. We have Cabinet documents, Royal Commission files and 
departmental records on defence, immigration, naturalisation, security, intelligence and many 
other issues involving the federal government. Each year thousands of people use the collection to 
undertake research. They include academics, genealogists, local historians, journalists, students, 
professional historians and lawyers. 

The Archives Act 1983 requires Commonwealth Government departments and agencies to transfer 
records that have significant and enduring value to the National Archives so that the public can 
have access to them. The Act provides for access to any Commonwealth records that are more 
than 30 years old (subject to certain exemptions). 

As you would appreciate the vast majority of records currently in our collection are paper files that 
predate the digital age. But how are we to ensure that the significant activities and decisions that 
the Commonwealth government is undertaking today, more often than not conducted and stored 
electronically, will survive or be accessible to the public thirty years from now? If we don t think 
about these issues now there will be a yawning gap in Australia's federal record. 

In 1998 the Australian Law Reform Commission described the state of recordkeeping in the 
Commonwealth as 'unregulated' and 'parlous'*. Indeed, over the past five years more than half the 
reports issued by the Australian National Audit Office have commented adversely on the quality of 
recordkeeping in government agencies.** 

This situation has been brought about by a variety of factors including changes in public service 
career paths and structures, 'multi-skilling', 'outsourcing', and the devolution of managerial 
responsibility. All of these trends have contributed to a loss of basic recordkeeping skills, 
procedures and systems and a greatly reduced awareness of the importance of good recordkeeping 
in the culture of the public service. 

So what has been the legacy of this cultural shift? 

The introduction of personal computers, networked office environments and new work practices 
combined with the demise of the Public Service Board, which used to issue recordkeeping 
guidelines and manuals to government agencies, has created a vacuum. Even when agencies know 
that they should be doing a better job managing corporate information they do not know where to 
turn for appropriate advice. 

Some years ago the National Archives decided that it had to take steps to fill this vacuum. It made 
a strategic decision to broaden the scope of its activities to become a standards and policy setter in 
the area of government recordkeeping, and more particularly, to address the challenges of 
recordkeeping in the digital age. We recognised that unless we got in at the start and helped 
agencies understand what records to create, how to create them and how to manage them over 
time there would be huge deficiencies in government accountability as well as a gap in the federal 
chronicle that we leave the next generation. 
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The Archives strategy has been to work in partnership with agencies to ensure good recordkeeping 
in support of efficient and accountable government. We set about producing the tools that agencies 
could use to tackle the problem. 

This work culminated in March this year with the release of a comprehensive suite of 
inter-dependent standards, policies and guidelines developed and published by the Archives on its 
website under the 'E-permanence' logo. 1 ' 1 Essentially, we want to work towards having government 
records, and especially electronic records, that are as reliable and durable as if they had been 
carved in stone. 

The basis of this new approach to recordkeeping is quite simple - recordkeeping should be 
managed systematically and never allowed to 'just happen'. It is based on AS4390 - the Australian 
Standard on Records Management - issued as a voluntary code of best practice by Standards 
Australia in 1996. ,v AS4390 was developed by representatives from public sector, corporate and 
academic institutions and is the, first national standard on records management to be adopted 
anywhere in the world. And I am pleased to say that it is now in the process of being adapted as 
an International Standard. 

Having given you a little background on the state of recordkeeping in the Commonwealth and the 
Archives response to it I'd now like to move on and talk about a couple of our standard setting 
initiatives designed to help agencies manage their digital information, including that subset that 
constitute federal records. 

The first of these initiatives concerns metadata. 

Metadata is often defined as 'data about data'. A more helpful definition might be 'structured 
information that describes and/or allows us to find, manage, control, understand or preserve other 
information over time'. v 

Of course metadata is simply a new term for the type of information that archivists, records 
managers and librarians have always collected to describe and control records and other 
information resources. Applying this new jargon, a traditional paper file is a 'metadata 
encapsulated object'. The file contains records and the file cover captures information about those 
records - what they are about, who used them, when they were used, what access restrictions 
apply to them and so on. In short, their context. 

There are two types of metadata that the National Archives has become actively involved in 
developing and promoting - resource discovery metadata and recordkeeping metadata. 
Recordkeeping is of course the Archives core business. But people often express surprise when 
they learn that the Archives has adopted a standards setting role in the online resource discovery 
area, as this is a field that is very often dominated by librarians rather than archivists. The Archives 
made a deliberate decision to play a leading role in this area for the following reasons. Firstly, 
government business is increasingly being conducted online and it is our job to ensure that 
adequate records, including adequate metadata of such transactions, are created. Secondly, with 
the trend towards making government information available online, it is certain that much of this 
information will also exist as records of one sort or another. If we are developing a recordkeeping 
metadata standard for government it makes no sense for there to be a totally separate and 
unrelated metadata regime for online resource description. As far as possible government should 
have an efficient, integrated metadata regime, and this may be difficult to ensure if different 
agencies are responsible for different, though related, government metadata standards. 

As far back as 1996 the National Archives was urging the government to give it the go-ahead to 
develop a resource discovery metadata standard that would help improve the visibility and 
accessibility of government information 3 nd services online. vi Our view was that success in this 
area could have spin-off benefits for some of the less visible areas of need (such as recordkeeping) 
by adding legitimacy and momentum to the Archives' policy initiatives. I am pleased to say that 
this is proving the case. 
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On 6 April this year, just a week after the launch of the Archives 'E-permanence' campaign, the 
government announced its Online Strategy.^ In relation to web-based publishing and service 
delivery, the strategy requires all government agencies to comply with metadata and 
recordkeeping standards issued by the National Archives. With respect to metadata, the standard 
identified in the strategy is the Australian Government Locator Service (or AGLS) metadata 
standard. 



AGLS is intended to standardise web-based resource descriptions and thereby help government 
search engines accurately and efficiently identify and retrieve web-based resources in response to 
public queries. The National Archives convenes a cross-jurisdictional expert group, the AGLS 
Working Group, which advises and assists it to develop and deploy the standard. viii The AGLS 
standard has now been endorsed by all Australian governments at Federal, State and Territory 
level. It is also attracting considerable interest in other national jurisdictions, most notably New 
Zealand, Canada and Great Britain. 



The AGLS standard is based on the Dublin Core resource discovery metadata standard, but extends 
that standard with the addition of four new descriptive elements and a number of new 
sub-elements or 'qualifiers'. These extensions reflect the particular requirements of Australian 
governments for precise online resource discovery. The four AGLS elements that have been added 
to the basic 15 elements of Dublin Core are: 

• function - that describes the function/s of government to which the resource relates; 

• availability - that provides information on how offline resources may be obtained; 

• mandate- that refers to the legislative or regulatory basis for the resource; and 

• audience - that describes the target audience for the resource. 



The set of 19 AGLS elements are listed below according to their descriptive purpose: 
• ownership and creators of the resource - creator , publisher , contributor, rights 



• intellectual content about the resource - title, subject *, description, source, language, 
relation, coverage, function *, audience, mandate 

• electronic or physical manifestation of the resource - date, type, format, identifier **. 
availability ** ix 



AGLS is designed to be simple, flexible and dynamic in its application. Only six of the 19 elements 
are mandatory (and these are underlined in the aforementioned list). AGLS metadata can be 
applied at the point at which an object is created or published online, and it can also be added to 
and improved as the particular resource evolves or changes over time. It can be used to describe 
single items or logical aggregations of resources. In addition, agencies can choose how much 
structure they wish to deploy in their metadata - they can choose to create simple, unqualified 
descriptions or they can create much richer descriptions using a range of different elements and 
qualifiers.* 



The Archives has published a user manual and application guidelines to help agencies understand 
and implement the AGLS standard. 



The inclusion of the 'function' element is one of the more significant features of the AGLS standard. 
This element establishes a robust link between the world of online resource discovery and the 
world of record keeping. Since the publication of the Australian Standard on Records Management 
the Archives has been recommending to agencies that they classify all their records by function and 
activity. 

'Functions' represent the major responsibilities that are managed by an agency to fulfil its goals. 
'Activities' are the major tasks performed by the agency to achieve its functions. There are often a 
number of activities associated with any one function. Each of these activities may involve one or 
more tasks or 'transactions'. 'Records' are often generated as a by-product and evidence of these 
transactions. 
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Efy way of example, 'publication' Is a common administrative function across government. Drafting, 
marketing, planning and tendering are some of the activities performed as part of the publication 
function. The process of drafting publications, for example, may Involve creating draft copies, 
distributing drafts for comment, receiving comments on drafts, evaluating comments and so on. 
Each of these tasks or transactions may generate records. 




By looking at their business this way, government agencies can better understand the context In 
which records are generated. They can then allocate an appropriate level of resources to manage 
their records according to the strategic significance they pose to the organisation. 

As part of the E-permanence campaign the National Archives Is encouraging all agencies to analyse 
their functions and activities and use this as an intellectual framework to develop a corporate 
thesaurus for file titling and retrieval purposes, indeed, for controlling all corporate Information 
resources. 

From a government resource discovery point of view business activities or 'functions' are also a 
useful means of resource classification. While there are a wide variety of different subject 
thesauruses In use in government, thus making It impossible to achieve national consistency in 
online resource classification, there Is only one comprehensive national functions thesaurus that Is 
recommended for use when creating AGLS metadata. 

This thesaurus, the Australian Governments Interactive Functions Thesaurus (or AGIFT), is a 
high-level thesaurus that covers the functions performed by all three tiers of government in 
Australia.* 1 It was developed by the National Archives in consultation with representatives from 
Commonwealth, State, Territory and local government, academics, information professionals and 
information managers from individual agencies. 

AGIFT aims to help people searching government websites to retrieve relevant results by linking 
natural language terms with their bureaucratic equivalents. It enables people to find Information 
about familiar subjects without needing to know or understand the functional structure of 
government. For example, search engines deploying the AGIFT thesaurus enable a user interested 
in government information on 'kindergartens' to retrieve information that has been classified 
according to the government term 'early childhood education'. At present the thesaurus supports 
plain English searching of some 300,000 terms and equates these to 26 over-arching terms that 
cover the functions of all three levels of government and a further 500 or so narrower and related 
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terms that describe these broad functions in more detail. 

The basic message that the Archives aims to promote is that if government agencies have gone to 
the effort of publishing information Or providing services via the Web, then it is worth linking those 
resources to some AGLS metadata and using functional terms that can be searched via AGIFT to 
help ensure that people can find them. xii 

Apart, from the AGLS metadata standard for resource discovery, the Archives has also developed a 
metadata standard for recordkeeping. Like resource discovery metadata recordkeeping metadata 
helps describe and locate information. But more importantly, it helps control and manage 
information in a way that preserves its integrity and authenticity and enables it to serve as 
evidence of business activity over time. Such evidence is essential for legal and public 
accountability purposes. 

Recordkeeping metadata is designed to: 

• uniquely identify records; 

• authenticate records; 

• document and preserve their content, context and structure over time; 

• administer conditions of access and disposal; 

a track the history of their use and management; 

• facilitate the transfer or migration of electronic records between IT systems; 

• restrict unauthorised use; and 

• help users find and understand records. 

As I mentioned earlier, this type of metadata is not new. What is new is the need for information 
professionals - records managers, archivists, IT specialists, data administrators and the like - to 
work together to ensure that the electronic systems we use to conduct our daily business have the 
capacity to create and capture this metadata. Unless electronic records are created and managed 
properly in well-designed systems that can guarantee their reliability, durability and accessibility, 
archivists are not going to have many records that they can preserve for long-term use or that will 
be worth preserving for long-term use. Nor will organisations have the type of records that are 
essential for the efficient, effective and accountable conduct of their business. 

The National Archives published its Recordkeeping Metadata Standard for Commonwealth Agencies 
in April 1999. xiii Like the other initiatives I have mentioned this standard was developed 
collaboratively - with software vendors, academics, government agencies and other experts. Its 
aim is to define the metadata that government agencies should capture in their corporate 
information systems so that these systems satisfy their needs for evidence over time. The National 
Archives recordkeeping metadata standard consists of 20 descriptive elements, eight of which are 
mandatory, and a further 65 'sub-elements' or qualifiers that add richness to the 20 main 
elements. 

In developing the Recordkeeping Metadata Standard, the Archives was determined to maximise the 
overlap between it and the AGL5 metadata standard. It recognised that a high proportion of 
resources that should be described using AGLS metadata would be records that are moved from a 
corporate recordkeeping environment into a web environment, or indeed records that exist 
simultaneously in both environments. Because both resource discovery metadata and 
recordkeeping metadata help people find resources in a networked environment, they have a lot in 
common. Of course, recordkeeping metadata aims to do a lot of other things as well. As such,, the 
Archives view has been that resource discovery metadata should be regarded as a subset of 
recordkeeping metadata. Under this unified regime, agencies need only create metadata once but 
can use it many times for different purposes, including records management and web-based 
information dissemination. 

By way of example, the table below illustrates the relationship between the eight mandatory 
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elements of the Recordkeeping Metadata Standard and elements of the AGLS standard. As you can 
see five of the six mandatory AGLS elements (underlined) correspond with the recordkeeping 
elements: 

• agent = creator , publisher , other contributor 

• rights management = rights 

• title = title 

• date = date 

• aggregation level = type (aggregation level qualifier) 

• record identifier = identifier 

• management history = date (part only) 

• disposal = [not applicable] 

Having developed and issued these inter-related metadata standards the major challenge now is to 
work with government agencies and the software vendor community to encourage their adoption 
and deployment, and, as far as possible, to find ways to automate their creation and management. 
This will require ongoing effort which in turn will necessitate regular reviews and revisions to the 
standards as we learn more about their practical application. As with any standard, our metadata 
standards have to continue to evolve to remain relevant and useful. 

Nevertheless, we are confident that the Commonwealth government now has a solid foundation of 
metadata standards upon which efficient and effective regimes for recordkeeping and online service 
delivery can be built. 

Apart from its promotion of metadata standards, the Government's Online Strategy also requires 
agencies to make and keep records that accurately document their public websites over time to 
satisfy business and accountability requirements and community expectations. 

Essentially this is nothing new - under the Archives Act Commonwealth agencies have legal 
obligations regarding the retention and disposal of Commonwealth records. Nonetheless, the 
dynamic nature of online activities presents special challenges. 

The first challenge is to recognise that websites are a form of publication (and are therefore 
records for archival purposes) and that websites may also generate records, particularly if they 
provide an interface for the provision of goods and services through the use of e-commerce. 

The second challenge is to decide what records agencies need to make of their web-based 
activities, when to make them, how to make them and how to keep them so that they remain 
accessible and meaningful as long as they are required. 

The Archives has developed a policy paper and guidelines to help agencies address these 
challenges. The policy paper is available on our website while the companion guidelines are in the 
final stages of drafting and are expected to be published by the end of the year. xiv 

Unfortunately we cannot offer agencies easy solutions to meet their recordkeeping responsibilities 
in the online environment. 

Websites today come in many different forms. These range from simple collections of static pages 
which display the same information to all visitors, through to pages that are created and displayed 
dynamically in response to specific queries. In addition many websites now do something - they 
enable visitor details to be captured, online orders to be taken, and personalised information to be 
displayed based on user profiles. The nature of an agency's site will influence its recordkeeping 
requirements. The technology that supports websites is also changing rapidly and the hardware 
and software that an agency uses today to make a record of its site may not be compatible with 
the tools it relies on tomorrow to access that record. 

In the Archives view, the effective management of web-based records relies on a systematic 
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approach that is generally applicable to all records, regardless of format. This approach is based on 
AS4390, the Australian Standard on Records Management. 

• firstly, agencies should actively take responsibility for recordkeeping; 

• secondly, recordkeeping should be based on a thorough understanding of the specific 
functions and activities performed by an agency and an understanding of the wider 
socio-political environment in which it operates; 

• thirdly, recordkeeping should be based on a systematic analysis of an agency's need to make 
and keep records relating to those functions; and 

• lastly, agencies should establish business systems and manage records in accordance with 
these identified needs. 

The methodology that we advocate to work through these generic recordkeeping issues is set out 
in more detail in Designing and Implementing Recordkeeping Systems: a Manual for 
Commonwealth Agencies, fondly known by the acronym DIRKS. It represents the backbone of the 
Archive's new approach to recordkeeping and is available on the Archives website. xv 

Using DIRKS in conjunction with the forthcoming Archiving Websites Guidelines will help agencies: 

• promulgate a policy on making and keeping web-based records; 

• assign and document specific responsibilities for recordkeeping practitioners, web 
administrators and IT staff; 

• determine what records the agency must create to satisfy business needs, broader 
accountability requirements and community expectations and assess and quantify the 
business risks of not maintaining full accountability of their actions; 

• assess technological options for creating web-based records that retain the format and 
functionality that give them meaning; and 

• explore storage and preservation issues to ensure that records of web activities remain 
accessible over time in accordance with identified requirements. 

The Archives cannot answer these questions for agencies. We can provide advice on the steps that 
agencies should take to address these issues and we can discuss the emerging technological 
options that may satisfy archival requirements. But ultimately agencies will need to establish 
partnerships between their recordkeeping professionals, IT staff such as network managers or data 
administrators, web administrators, and corporate governance personnel such as auditors to work 
through these issues and develop solutions most appropriate to their business needs and broader 
accountabilities. 

In closing, I hope that this brief presentation has given you an understanding of the Archives role 
in ensuring that present and future generations have proper access to the services and records of 
government in the online era. Our intention has been to develop a range of interrelated standards, 
policies and guidelines that provide government agencies with the intellectual framework to 
manage their information resources, including their federal records, in an integrated way. 

The proof of our success will be the existence of durable, meaningful and accessible electronic 
records in the years to come. 



Footnotes 



i Australian Law Reform Commission, Australia's Federal Record: A Review of the Archives Act 
1983, Sydney, 1998, p. 27. 

ii Australian National Audit Office reports can be found at http://www.anao.gov.au/ 

iii The E-Permanence website can be found at http://www.naa.gov.au/recordkeeping/ 
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iv Standards Australia, Records Management AS 4390, Homebush, 1996. This standard forms the 
basis of the draft international records management standard, ISO 15489. 

v Coined by Adrian Cunningham in 'Six degrees of separation: Australian metadata ini natives and 
their relationships with international standards' delivered to the Netherlands Institute for Archival 
Education and Research in June 2000. 

vi Information Management Steering Committee, Management of Government Information as a 
National Strategic Resource; Report on Information Management in the Commonwealth 
Government, Canberra, 1996, pp. 242-245. Available at 
http://www.dcita.gov.au/ogo/imsc/imscrept.htm 

vii The website for the Government Online Strategy can be found at http://www.govonline.gov.au/ 

viii The Australian Government Locator Service home page can be found at 
http://www.naa.gov.au/recordkeeping/gov_online/agls/summary.html 

ix * and ** denotes alternative mandatory elements 

x The AGLS standard is defined in the Australian Government Locator Service Manual for Users, 
available at http://www.naa.gov.au/recordkeeping/gov_online/agls/user_manual/intro.html; while 
practical advice on applying the standard can be found in Applying the Australian Government 
Locator Service (AGLS) Metadata Standard in Commonwealth Agencies, Version 1.0, August 2000, 
available at http://www.naa.gov.au/recordkeeping/gov_online/agls/guidelines/intro.html 

xi Information about the Australian Governments Interactive Functions Thesaurus can be found at 

http://www.naa.gov.au/recordkeeping/gov_online/agift/summary.html 

xii Of the 116 agencies that provided authorised responses to OGO's first reporting round in July 
2000 70% expect to have obtained National Archives accreditation to use the AGLS logo on their 
websites by 31 December 2000. 85% expect to have approval by 31 December 2001. See Office 
for Government Online, Government Online Round One Survey Results, September 2000 available 
at http://www.govonline.gov.au/projects/strategy/GovernmentOnlineSurvey.htm 

xiii National Archives of Australia, Recordkeeping Metadata Standard for Commonwealth Agencies 
Version 1.0, May 1999. Available at: 

http://www.naa.gov.au/recordkeeping/control/rkms/summary.htm 

xiv National Archives of Australia, Archiving Websites: A policy for keeping web-based records in 
the Commonwealth Government, March 2000 is available on our website at 

http://www.naa.gov.au/recordkeeping/er/web_records/intro.html 

xv Designing and Implementing Recordkeeping Systems: Manual for Commonwealth Agencies, 
which was jointly developed by the National Archives of Australia and the State Records Authority 
of New South Wales, and published as an exposure draft in February 2000 can be found at 

http://www.naa.gov.au/recordkeeping/dirks/summary.html 
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